test: Avoid hang in big segment status polling specs - #440
Merged
Merged
Conversation
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor
|
@cursor review |
kinyoklion
marked this pull request as ready for review
September 15, 2026 15:44
jsonbailey
approved these changes
Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the intermittent JRuby CI hang by removing a race in the big segment status polling specs, and adds a job timeout so a future hang fails fast instead of running until GitHub's limit.
spec/impl/big_segments_spec.rb: the twostatus pollingexamples no longer assume the first status change is observed afteradd_observer.github/workflows/build-gem.yml:timeout-minutes: 20on the gem build jobRequirements
Implementation details
Root cause
BigSegmentStoreManager.newstarts its status poll worker immediately (start_delayof 0), andBigSegmentStoreStatusProviderImpl#update_statusonly notifies observers when the status differs from the previous one. Bothstatus pollingexamples constructed the manager, then added an observer, then blocked onQueue#popfor the initial status. If the worker's first poll ran beforeadd_observer, the initial status was already recorded, no notification was ever sent, and the unboundedQueue#popblocked forever — taking the whole rspec process with it, so RuboCop and the gem build never ran and the job sat until it was cancelled at the 6 hour limit.MRI generally loses that race (a newly spawned thread does not run until the main thread yields), which is why this only showed up on the JRuby job, where threads run truly concurrently.
Solution
The examples now read the initial status directly from
status_provider.statusand use a helper that waits, with a timeout, for the next status matching a predicate. That makes them independent of whether a status was queued before the observer was registered, and a broken expectation now fails instead of hanging.The job timeout is defense in depth: any other future hang in tests, rubocop, or contract tests now fails the job in minutes.
Alternatives considered
Changing the manager to delay the first poll or to always notify on the first status would make the specs pass, but that changes SDK behavior to accommodate a test, and the current behavior (no notification unless the status changes) is intended.
Testing
Reproduced the hang locally on JRuby 9.4.15.0 (2 pinned cores), where the
status pollinggroup hung every attempt before the change and passed repeatedly after it. The full suite (bundle exec rspec spec --tag '~flaky') passes on JRuby with the persistent stores running.Known remaining issue (not addressed here)
ld-eventsourceconnection threads can still outlive an example and call into an expired rspec double viaStreamProcessor#log_connection_result, which prints anExpiredTestDoubleErrorfrom a terminating thread. It is noisy but does not hang or fail the suite; worth a separate look.Link to Devin session: https://app.devin.ai/sessions/700251d712274fcfaabc41ae8e479b3f
Open in Devin Desktop: https://app.devin.ai/desktop/session/700251d712274fcfaabc41ae8e479b3f?variant=devin
Requested by: @kinyoklion
Note
Overview
Fixes intermittent JRuby CI hangs in big segment status polling tests and caps gem build job runtime so future stalls fail fast.
The two
status pollingexamples no longer block on the firstQueue#popafter registering an observer—a race on JRuby where the poll worker can finish before the observer is attached, so no notification fires and the test waits forever. They now assert the initial state viastatus_provider.statusand use a newnext_status_matchinghelper (5s timeout, predicate filter) for subsequent transitions.The build-gem workflow job gets
timeout-minutes: 20as defense in depth.Reviewed by Cursor Bugbot for commit c9aac8f. Bugbot is set up for automated code reviews on this repo. Configure here.